Processes for removing noise from data or a signal are generally referred to as denoising, and a number of denoising processes have been developed The goal of most denoising processes is to convert a noisy signal SN into a recovered signal SR that is “closer” to an input or clean signal SC. Since the clean signal SC is unavailable during denoising, a denoising process may use a priori knowledge of characteristics of the clean signal SC when attempting to identify and remove noise. Alternatively, universal denoising processes in general operate with little or no a priori knowledge of the characteristics of the clean signal SC but may instead rely on knowledge or measurements of the source of the noise.
Discrete Universal DEnoising, sometimes referred to herein as DUDE, is a type of universal denoising that can be applied to a noisy signal SN containing discrete symbols si from a known alphabet A={α1, α2, . . . αn}. DUDE uses a context η(s) of each symbol s being denoised, where the context η(s) in general is set of symbols that in some sense neighbor the symbol s. For example, in a signal composed of a series of symbols representing digitized amplitudes of audio or another continuous signal, a context η(s) of a symbol s may include one or more symbols preceding and/or following symbol s in a time ordering of noisy signal SN. In an image composed of symbols that are pixel values, each symbol s corresponding to a pixel away from the edge of the image may have a context η(s) defined to be the pixel values for pixels within a 2-dimensional area in the image.
DUDE generally includes a modeling scan and a replacement scan of the noisy signal SN. The modeling scan determines count vectors m(η) for respective contexts η. Each component m(η)[i] of a count vector m(η) indicates the number of times a corresponding symbol αi is found in context η in the noisy signal SN. Related count vectors q(η) containing counts of symbols in the clean signal SC that end up having context η in the noisy signal SN indicate the input distribution of the clean signal SC. Lacking the clean signal SC, DUDE cannot directly determine the count vectors q(η) but can estimate count vectors q(η) using available information from the noisy signal SN and assumptions concerning or knowledge of the source of targeted noise. In particular, the source of the targeted noise can be modeled as a channel that is represented by a channel matrix Π, and one technique for estimating the input distribution is through vector multiplication of measured count vector m(η) by the inverse of channel matrix Π, e.g., qT(η)≈mT(η)Π−1. (As a convention for matrix expressions herein, vectors such as q(η) and m(η) are assumed to be column vectors, and transposed vectors such as qT(η) and mT(η) are assumed to be row vectors.)
The replacement scan generates recovered signal SR by replacing each occurrence of a symbol αi found in context η in the noisy signal SN with a symbol αj that minimizes an estimated distortion resulting from such replacement of symbol αi with symbol αj relative to the clean signal SC. (Symbols αi and αj may be equal, which is equivalent to the replacement scan leaving αi unchanged when found in context η.) Equation 1 indicates a distortion D(αi,αj,η) suitable for the DUDE process and an estimation of the distortion using estimated count vector mT(η)Π−1 in place of count vector qT(η). In Equation 1, vector λ(αj) is a column of a symbol transformation matrix Λ, and each component of vector λ(αj) indicates a measure of the distortion created in clean signal SC by replacing the symbol corresponding to that component with symbol αj. Vector π(αi) is a column of channel matrix Π and has components indicating the probability of the channel (i.e., the source of targeted noise) replacing a corresponding symbol with symbol αi, and the Schur product λ(αj)⊙π(αi) generates a vector having components equal to the products of corresponding components of vectors λ(αj) and π(αi).D(αi,αj,η)=qT(η)(λ(αj)⊙π(αi))≈mT(η)Π−1(λ(αj)⊙π(αi))  Equation 1
A paper by Tsachy Weissman, Erik Ordentlich, Gadiel Seroussi, Sergio Verdú, and Marcelo J. Weinberger, entitled “Universal Discrete Denoising: Known Channel,” IEEE Transactions On Information Theory, Vol. 51, No. 1, January 2005 provides further description of the DUDE process and is hereby incorporated by reference in its entirety.
Denoising using DUDE and an estimated distortion mT(η)Π−1(λ(αj)⊙π(αi)) will provide best performance if the quantity mT(η)Π−1 provides an accurate estimate of the count vectors qT(η). However, for some types of noisy channels, e.g., channels producing additive Gaussian noise, channel matrices Π can be near singular so that numerical determination of the inverse matrix Π−1, e.g., using computers, results in significant errors. Additionally, the measured count vectors m(η) are subject to statistical variations that can be magnified when multiplied by inverse matrix Π−1 whether inverse matrix Π−1 is accurate or not. Accordingly, other methods for estimating the input distribution are needed when denoising signals that were subjected to some sources of noise.